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ABSTRACT 


Graduate teaching assistants (GTAs) often teach the majority of contact hours within engineering 
with little pedagogical knowledge. To plan efficient professional development for GTAs to address 
this, we created and administered a survey to measure the perceived importance of GTAs’ roles and 
responsibilities. GTAs, faculty, and students rated the importance of each of 24 GTA roles and respon¬ 
sibilities on a 5-point Likert scale and the perceived competence of GTAs on these 24 items. We used 
exploratory factor analysis and reliability tests to ascertain construct validity and reliability. We con¬ 
ceptualized the resulting four factors (which accounted for 54% of the total variance) as the following: 
1) clear communication, 2) student management, 3) preparation for feedback and assessment, and 
4) course management/policy knowledge. Correlations with an empirically-validated teacher motivation 
measure indicated the four factors were positively related to GTAs’ intrinsic motivation. Comparisons 
among scores for GTAs, faculty, and students revealed many similarities and several significant differ¬ 
ences among these groups, especially in ratings of competence between GTAs and others. 

Key Words: Graduate Teaching Assistants; GTA Training and Development; Faculty, student, and 
GTA perspectives 


INTRODUCTION 


A widely-acknowledged factor inhibiting the learning, development, and retention of engineer¬ 
ing students is the inexperience and lack of pedagogical education of many instructors (Moving 
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Forward to Improve Engineering Education 2007). These concerns about recruitment and retention 
of students in engineering disciplines have resulted in numerous calls for reform (Rugarcia et al. 
2000; Educating the Engineer of 2020: Adapting Engineering Education to the New Century 2005; 
Galloway 2007). Regardless of the chosen response to such calls, it is clear that quality education 
requires the presence of instructors who have learned to teach effectively. Unfortunately, because we 
often rely on “on-the-job” training, faculty become skilled at teaching after receiving their doctoral 
degrees and “practicing” on students. For this reason, institutions commonly establish teaching 
effectiveness centers dedicated to faculty development. Moreover, and of greater concern to us, 
much undergraduate teaching, especially during laboratories which may constitute 50% or more of 
the time that students are in the classroom, is performed by Graduate Teaching Assistants (GTAs) 
who may receive no training in pedagogy prior to their first teaching experience (Cox et al. 2011; 
Shannon, Twale, and Moore 1998). Given the importance of instruction in determining students’ 
willingness to pursue undergraduate degrees in engineering (Seymour and Hewitt 1997) and the 
critical role the first year laboratory can play for students making the transition from high school 
to college (O’Neal et al. 2007), the pedagogical knowledge and skills of GTAs becomes even more 
critical. Sadly, while one would predict that the inexperienced would be aware of their lack of skill 
and pedagogical knowledge and would seek assistance, this does not always appear to be the case. 
Perhaps this indicates that GTAs are performing at the expected level for their profession. Alterna¬ 
tively, this may indicate ignorance of expectations or lack of interest or motivation. 

We conducted the research described here to ascertain, through surveys of faculty, GTAs, and 
students, what characteristics are considered to be most important in successful engineering GTAs 
and what are the perceptions of GTAs’ competency levels. This study was undertaken in prepara¬ 
tion for the development and implementation of a GTA training program, both to set the objectives 
of the program and to establish a baseline for measuring the impact of the program in the future. 
We tested psychometric properties of the measure by examining factor structure, reliabilities, and 
concurrent construct validity. To examine concurrent construct validity, we examined how our 
new survey is related to teacher motivation measure, an empirically validated measure. This paper 
presents the results for the reliability and validity tests, as well as the insights we gathered from 
administering the survey at our institution. 

Additional sections of the paper include Background and Rationale, providing justification for GTA 
training, insights from previous research on training GTAs, and common problems faced by GTAs. 
The Methods section provides details on the survey participants, the components of the survey, 
teacher motivation measure, and procedures. The Results and Discussion sections provide results 
for the factor analysis and other reliability and validity tests, and an analysis of the data from the 
trial administration of the survey. The Conclusions, Limitations, and Future work section summarizes 
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the findings, lists the limitations and describes how the results may be used to develop or improve 
a training program for GTAs. 


BACKGROUND AND RATIONALE 


It is logical to think that variations within courses (i.e. across sections or semesters) can affect 
students and programs. When engineering courses are taught by different faculty members and 
graduate teaching assistants using different teaching approaches, students’ course satisfaction and 
learning outcomes vary considerably among courses. While oversight of course content often exists, 
students taking the same class under different instructors often have significantly different levels of 
conceptual understanding and skills gained from that course. This can impact subsequent courses 
that require pre-requisite knowledge. Variations in teaching quality among faculty can be addressed 
through professional development programs that introduce instructors to the best practices and latest 
developments in pedagogy and course design. However, although graduate teaching assistants play 
an important role in engineering student learning, many higher education institutions do not have a 
professional development program for GTAs in place (Luft et ai. 2004). Therefore, another solution 
to the variability in student understanding and satisfaction among sections or semesters might be 
to provide GTAs with the same professional development training, so labs and discussion sections 
are taught well and in a consistent manner. In an effort to enhance GTAs teaching effectiveness, 
Cox and her colleagues (Cox et al. 2011) developed an assessment tool (i.e., the Global Real-time 
Assessment Tool for Teaching Enhancement) to provide GTAs quantitative and qualitative feedback 
regarding their instructional performance from students and supervisors. In conjunction with the 
beneficial feedback information from various stakeholders, GTAs’ reflection and perception of their 
instructional practices play an important role in engaging in professional development endeavors 
to enhance their teaching skills. While extensive GTA training and professional development pro¬ 
grams have their merits (Bond-Robinson and Rodriques 2006; Prieto, Yamokoski, and Myers 2007), 
there exists the question about whether programs less costly in time or money could be sufficiently 
beneficial. To test this, one starting point would be with an evaluation of the expectations for GTAs 
and the degree to which they meet expectations without training. 

As the nation tries to improve Science, Technology, Engineering, and Mathematics (STEM) education 
on many different fronts, work focusing on GTA training must not be overlooked. At doctoral degree 
granting institutions, GTAs may contribute substantially to undergraduate education (Sundberg and 
Armstrong 1993). Many engineering courses have large lab components predominantly taught by 
GTAs. For a typical 3 credit-hour engineering class at many institutions, faculty teach two 50-minute 
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lectures, while less experienced GTAs teach two hours of lab per week. Because GTAs interact as much 
with students as do faculty, training in classroom management and pedagogy for these novice instruc¬ 
tors may have greater impact on student learning and retention than similar efforts aimed at more 
experienced faculty, who may be set in their ways or may already be more accomplished instructors. 
A training program may reduce the negative impact of having GTAs who do not consider themselves 
teachers, or who are not aware of their roles and responsibilities. Because GTA selections are often 
influenced by advisors’ needs for their research programs and students seeking financial assistance 
for tuition, irrespective of whether they are qualified or prepared to be GTAs, teaching quality among 
beginning GTAs can be extremely uneven. GTAs often receive no or little systematic training (Buerkel- 
Rothfuss and Gray 1990; Luo, Bellows, and Grady 2000; Prieto 1999; Prieto and Altmaier 1994), and 
are unprepared to take on their teaching responsibilities due to lack of exposure to the best practices 
in teaching and classroom management (Anderson 1992; Civikly and Hidalgo 1992). At the institution 
under study, there exists little coordination of the selection and training of GTAs beyond requirements 
for international GTAs to pass an exam and receive some training on culture and communication, and 
safety training for all GTAs. 

In science and engineering courses, there is a long history of using homework, labs, and 
discussion sections as a means of providing students with the opportunities to practice and 
assess their conceptual understandings and higher order cognitive skills. At universities with 
substantial graduate programs, this is the primary domain of the GTAs. Thus, GTAs become a 
substantial source of feedback and the primary contact for individualized instruction (Herrington 
and Nakhleh 2003). Laboratories provide opportunities for “hands-on” experience favored by 
concrete, transitional and sensory learners (Leonard 1997). While laboratories help students 
learn to make data-driven decisions (Pickering 1988), their primary goal in the minds of faculty 
is to afford students more time to learn concepts (Bond-Robinson and Rodriques 2006). The 
laboratory environment and the influence of the GTA on that environment have been shown 
to play a major role in student retention in STEM disciplines (O’Neal et al. 2007). Given the 
increased responsibilities of GTAs and their impact on student learning, preparing GTAs to be 
effective teachers is critical not only in retaining undergraduates and improving student learning 
and engagement, but also in retaining qualified college instructors. Many new undergraduate 
engineering students make decisions related to their future academic endeavors based on their 
perceptions of those GTAs. Moreover, the GTAs, if unprepared and untrained as teachers, will 
be more likely to experience frustration and failure (Allen and Rueter 1990), which may affect 
their own decision to leave academia. 

Even with training programs, new GTAs still consider college teaching a challenge (Feezel and 
Myers 1997) when their pressing concerns in relation to teaching are not appropriately addressed 
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in the GTA training program. Thus, identifying and incorporating concerns of GTAs into the devel¬ 
opment program is a key in enhancing the effectiveness of the training programs for GTAs. Fuller 
(Fuller 1969) suggests that to ensure effective teacher development programs, it is critical to assess 
teacher concerns accurately. In addition, teacher training or professional development programs 
that do not reflect the needs and interests of participants are unlikely to motivate them, which in 
turn can result in the failure to attain the program’s educational goals and objectives (Cho et al. 
2011). This speaks directly to the importance of needs assessment surveys designed to identify what 
motivates and concerns teachers in advance of developing training programs. The present study 
focuses on developing a GTA needs assessment survey and investigating its psychometric proper¬ 
ties to provide empirical evidence that the survey is a valid and reliable tool that can be used to 
capture areas for GTA training. 

Given that the primary goal of designing and administering GTA training is to enhance 
teaching efficacy and effectiveness, literature on teacher efficacy guided us to define areas for 
teaching improvement that need to be the potential focus of GTA training. Previous research has 
conceptualized three primary domains of teaching in which efficacious teachers tend to dem¬ 
onstrate excellence (Klassen et al. 2009; Tschannen-Moran and Hoy 2001). The three domains 
include 1) student engagement (i.e., capabilities to motivate and engage students to learn), 
2) instructional strategies (i.e., capabilities to employ a variety of appropriate instructional and 
assessment strategies), and 3) classroom management (i.e., to manage student behaviors and 
classroom discipline). Based on these domains of teaching excellence, we developed survey 
items to assess the degree to which GTAs place a value on and feel competent and prepared 
to perform successfully. 

To further examine the construct validity of the new measure, we examined how GTAs’ motiva¬ 
tion to teach is related to their responses on the survey measure. We expected that the extent to 
which GTAs are motivated in a self-determined manner will be associated with their perception 
regarding the importance of GTA roles and responsibilities. Deci and Ryan (2000) discussed that 
different types of self-determined motivation can be assessed on a continuum ranging from intrinsic 
to identified to introjected to extrinsic to amotivation. Intrinsic motivation represent the highest 
level of self-determination while amotivation represents the lowest level of self-determination. 
People with intrinsic motivation engage in a task because of their interest in or enjoyment of the 
task whereas those with identified motivation engage in a task because of the value and impor¬ 
tance of the task. People with introjected motivation tend to engage in a task because of internal 
pressure and responsibilities whereas those with extrinsic motivation engage in a task because of 
external forces, such as monetary rewards or external pressure. Amotivation denotes no willing¬ 
ness to engage in a task. 
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METHOD 


Participants 

The study was conducted at a land-grant institution located in the Midwest. Invitations to partici¬ 
pate in the online survey were sent out via email to all engineering graduate students (N = 900) at 
the institution. Email lists were provided by the Office of Institutional Research, and invitations were 
sent so that the identity of each invitee was kept anonymous. We were interested in capturing the 
views of all graduate students (GTAs or not) in this project. However, for this study we used a subset 
of the participants who either currently work as GTAs or have done so in the past. Out of a total of 
216 students who responded to the survey, 144 students met the above criteria. Of the 144 graduate 
teaching assistants, 85.6% were male, 10.0% were female, and 4.4% did not report. Furthermore, 
66.3% were master’s degree students and 33.7% were doctoral students. Although the proportion 
of female participants in our sample is significantly smaller than that of male participants, it actually 
represents the male/female ratio (81.5% male students, 18.5% female students) in the engineering 
student population of the participating institution. Of the respondents, 60% had an undergraduate 
degree from a non-US institution. The GTAs spanned the entire teaching experience range of teach¬ 
ing assistants, from those who were first time GTAs to those who had taught up to 12 semesters. All 
results provided below are for these 144 survey respondents. 

Invitations for research participation were also sent out to all the faculty and the undergradu¬ 
ate students in the college of engineering. A total of 38 (out of about 100) faculty and 365 (out of 
about 2000) undergraduate students responded to our survey. Demographic information was not 
collected from these groups, as it was not deemed necessary. Email lists for these groups were also 
provided by the Office of Institutional Research, and invitations were sent so that the identity of 
each invitee was kept anonymous. 

Procedure 

The survey was administered using SurveyMonkey (www.surveymonkey.com) and consisted of 
six sections: information required for informed consent, Needs Assessment Survey Importance 
Ratings (24 items), Needs Assessment Survey Competence Ratings (24 items), Teacher Motiva¬ 
tion Survey (15 items), demographic data regarding personal characteristics and career plans 
(10 items), and demographic data regarding employment specifics (10 items). Participants could 
skip any section or question. We offered participants the opportunity to add comments at the end 
of each opinion section. Faculty and undergraduate students were asked to complete two sections 
(Need Assessment Survey Importance Ratings and Competence Ratings), while GTAs were asked 
to complete all six sections. 
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Measure 

Needs Assessment Survey 

We developed a needs assessment survey to capture to what extent GTAs rate the importance 
of typical GTA roles and responsibilities such as providing feedback to students, grading student 
work in a fair and consistent way, and establishing a working relationship with students, etc. All 
roles and responsibilities chosen are provided in Table 1 of the results. The survey items were drawn 
from the literature or created by the authors as explained in the previous section. The instrument 
was developed to incorporate the following features: 

• Relevant and applicable in GTA training settings 

• Inclusive and comprehensive to capture GTAs’ perceptions and needs for professional devel¬ 
opment in higher education settings 

• Multi-dimensional (e.g., instructional practices, classroom management, engagement with 
students) 

The participants were asked to rate the importance they placed on each of 24 GTA roles and 
responsibilities on a 5-point Likert scale with 1 representing ‘Not at all important’ and 5 represent¬ 
ing 'Critically important’ on the roles and responsibilities questionnaire. All three groups (GTAs, 
faculty and students) rated all the items on the same scale. A 5-point Likert scale was also used for 
competence ratings for the 24 items with 1 representing ‘Not at all competent' and 5 representing 
‘Very competent.’ 

Analysis of the data included exploratory factor analysis and reliability tests to ascertain the 
validity and reliability of the survey using SPSS. Exploratory factor analysis was conducted with 
oblimin rotation with the 24 perceived importance items of GTA roles. We used the eigenvalue- 
greater-than-one rule (Kaiser and Rice 1974) and scree test (Cattell 1966), to determine the 
number of factors that would represent appropriately the perceived importance of GTA roles 
and responsibilities. Eigenvalues indicate the total variance accounted by each factor and fac¬ 
tors with eigenvalues equal or higher than 1 were retained. The scree test creates a plot of the 
eigenvalues by ordering each factor by variance and factors above elbow points in the plot (i.e. 
where sudden decrease in variance is observed) were retained. 

Motivation to Teach 

The Work Tasks Motivation Scale for Teachers (WTMST) (Fernet et al. 2008) was used for 
this study. The WTMST was designed to assess five constructs in motivation toward six different 
aspects of teachers’ work tasks: class preparation, teaching, evaluation of students, class man¬ 
agement, and administrative tasks. We made minor modifications of the original survey items to 
assess GTAs’ motivation to teach. The original scale include five subscales: 1) Intrinsic motivation, 
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2) Identified motivation, 3) Introjected motivation, 4) Extrinsic motivation, and 5) Amotivation. 
We chose this measure to further examine the construct validity of GTA needs assessment survey 
because it is one of the empirically validated measures of teaching beliefs. Each item starts with 
a STEM question, “Why do you teach?” Sample response items included “Because I find teaching 
interesting to do” for intrinsic motivation, “Because it is important for me to carry out this task” 
for identified motivation, “Because I would feel guilty not doing it” for introjected motivation, 
“Because the school obliges me to do it” for extrinsic motivation, and “Because I don’t know, 
sometimes I don’t see its purpose” for amotivation. Participants chose the rating that most closely 
corresponded to their reasons for teaching (Strongly disagree, Somewhat disagree, Neutral, 
Somewhat agree, and Strongly agree). We conducted an exploratory factor analysis to evaluate 
the construct validity of the modified version of the WTMST using GTAs’ self-report data. The 
result revealed a clear five-factor structure comprising intrinsic motivation, extrinsic motivation, 
and amotivation. The five factors accounted for 73% of the total variance and showed accept¬ 
able reliability coefficients (a = .79 for intrinsic motivation, a = .67 for identified motivation, a = 
.79 for introjected motivation a = .66 for extrinsic motivation, a = .89 for amotivation). We used 
the WTMST to evaluate the concurrent validity of our Needs Assessment Survey using SPSS to 
calculate and then to examine correlations between the factors identified in the factor analysis 
and the teacher motivation measure. 


RESULTS 


The results section is divided into two subsections. We first present the results on the reliability 
and validity analysis of the survey, including comparison with the WTMST. In the next subsection 
we present an example of what can be learned from administering the survey, i.e. the results on 
discrepancies between the importance and competence ratings by the three participant groups 
(GTAs, students, and faculty). The study on reliability and validity was presented at the 2011 ASEE 
National Conference (Sohoni, French, and Cho 2011). A study on the discrepancies was presented at 
the 2010 ASEE Midwest Section meeting (Cho, Sohoni, and French 2010), for a pilot sample. The pilot 
study did not ask for any demographic information, nor did it require GTAs to fill out the WTMST. 
While the number of participants in all three groups were lower than those in the study presented 
here, the results are consistent. In particular, the comparison of GTA ratings of competence with the 
students’ and faculty’s rating of GTA confidence showed the same trend in both studies. The results 
reported below are based on the data we collected for the follow-up study. 
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Factor 



i 

2 

3 

4 

IMP 13: Effectively communicating with students 

.81 




IMP 14: Explaining contents clearly 

.77 




IMP11: Being familiar with the course materials 

.63 




IMP6: Engaging the students with the learning material 

.62 




IMPS: Offering feedback on work to the students 

.55 




IMP15: Overcoming cultural and language conflicts 

.50 




IMP22: Maintaining authority over dominating and aggressive students 


.72 



IMP 17: Motivating inattentive/uninterested students 


.70 



IMP24: Managing students’ disruptive classroom behavior 


.67 



IMP23: Avoiding offending a student while trying to simplify a concept 


.65 



IMP 16: Speaking to the class publicly 


.64 



IMP21: Facilitating positive team dynamics and discussions 


.57 



IMP 19: Knowing what is expected of the TA 



.65 


IMP 12: Grading student work in a fair and consistent way 



.57 


IMP 18: Knowing answers to student questions (course content) 



.57 


IMP20: Getting students to think for themselves & learn to solve problems 



.55 


IMP 10: Treating students with compassion and respect 



.42 


IMP2: Being familiar with the syllabus 




.68 

IMP3: Being familiar with course objectives 




.65 

IMP7: Making your grading rubric available to the students 




.59 

IMP4: Preventing academic dishonesty 




.51 

IMP5: Dressing appropriately 




.49 

IMP9: Holding regular office hours 




.36 

IMP 1: Establishing working relationship with students 




.34 

Eigenvalue 

8.39 

1.97 

1.40 

1.22 

% of Variance Explained 

34.95 

8.22 

5.79 

5.07 

Total Variance Explained 



54.03% 



Note. Factorl=Clear Communication, Factor2=Student Management, Factor3=Preparation for Feedback and Assessment, 
Factor4=Course Management/Policy Knowledge, IMP=Importance 

Table 1: Factor Loadings for Needs Assessment Survey for GTA Training. 


Study of Validity and Reliability 

To assess the construct validity of the measure, an exploratory factor analysis was performed us¬ 
ing the principal component analysis. Results yielded four factors with eigenvalues greater than 1, 
accounting for 54% of the total variance (see Table 1). The scree test further suggested four factors 
that accounted for 35%, 8%, 6%, and 5% of the variance, respectively. Most survey items showed 


WINTER 2013 


9 












ADVANCES IN ENGINEERING EDUCATION 

A Survey to Capture Needs Assessment for GTA Training 




Importance 

M 

SD 

Reliability 

Alpha 

1 . 

Clear Communication 

4.35 

.55 

.81 

2. 

Student Management 

3.84 

.76 

.82 

3. 

Preparation for Feedback and Assessment 

4.42 

.54 

.77 

4. 

Course Management/Policy Knowledge 

4.05 

.59 

.77 


Table 2: Means, Standard Deviations, and Reliabilities of Four Factors. 


factor loadings greater than 0.40 except two items with a loading of 0.36 and 0.34, respectively and 
the cross-loadings were not substantial (all were less than 0.30). The first factor consisted of 6 items, 
reflecting GTAs' perceived importance of their ability to effectively communicate with students and 
explain course contents clearly, and thus was named ‘Clear Communication.’ The second factor com¬ 
prised 5 items reflecting GTAs’ role to deal with student behaviors and classroom disruption issues 
as well as to engage and motivate inattentive/uninterested students, and thus was named ‘Student 
Management.’ The third factor contained 5 items that capture GTAs’ role to prepare themselves for 
feedback and assessment. The last factor comprised 7 items that tap into GTAs’ responsibilities to 
become familiar with the course objectives and syllabus and holding office hours, so we named it 
‘Course Management/Policy Knowledge.’ Survey items and factor loadings are provided in Table 1. 

The data provided favourable evidence for internal consistency of the measure. The Chronbach 
alpha reliability coefficients with the sample of this study for the four factors was 0.81 (Clear Com¬ 
munication), 0.82 (Student Management), 0.77 (Preparation for Feedback and Assessment), and 
0.77 (Course Management/Policy Knowledge). 

Table 2 shows descriptive statistics (means, standard deviations, and reliabilities) of the four 
factors. GTAs placed higher importance on the factors of ‘Clear Communication’ and ‘Preparation 
for Feedback and Assessment’ than the other two factors. 

We evaluated the concurrent validity of the measure by examining correlations between the four 
factors and teacher motivation measure which has been empirically validated in the literature. The 
results revealed that the four factors were positively correlated with GTAs’ intrinsic motivation to 
teach (r =.18-.32) and identified motivation (r =.28-.34), while they showed little or no relations with 
introjected motivation (r =.03~.18), extrinsic motivation or amotivation (see Table 3). 

Results for Importance and Competence Comparisons 

The second purpose of our study was to capture varying perspectives and needs about engineer¬ 
ing GTA training from different groups involved in engineering education (i.e., GTAs, faculty, and 
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i 

2 

3 

4 

5 

6 

7 

1. Clear Communication 

_ 







2. Student Management 

.53" 

_ 






3. Preparation for Feedback and Assessment 

.66** 

.60" 

_ 





4. Course Management/Policy Knowledge 

.66** 

.52" 

.58" 

_ 




5. Intrinsic Motivation To Teach 

.22" 

.32" 

.18* 

.23" 

_ 



6. Identified Motivation to Teach 

.30" 

.34" 

.29" 

.28" 

.65" 

_ 


7. Introjected Motivation to Teach 

.03 

.18* 

.03 

.04 

.07 

.20* 

_ 

6. Extrinsic Motivation To Teach 

-.02 

-.04 

.01 

.04 

-.15 



7. Amotivation 

-.05 

.01 

-.09 

.05 

-.17* 

.47** 



*p < .05, **p < .01 

Table 3: Correlations between Factors Extracted from Needs Assessment Survey and 
Teacher Motivation. 


students). We asked all participants to rate the GTAs’ competence and importance of GTA roles 
on each of the 24 survey items. Mean scores were calculated for each of the four categories that 
emerged from the factor analysis. 

Difference among Groups in Importance Ratings 

Table 4 shows the overall comparison of the average ratings for the four categories by the three 
groups on importance and competence. A multivariate analysis of variance (MANOVA) revealed signifi¬ 
cant differences among the three groups’ competence and importance ratings (F = 7.23, p < .001). 

On the importance ratings, subsequent univariate analysis of variance (ANOVA) indicated overall 
significant differences among the three groups in all categories (F = 4.31, p < .05 for 'Clear Commu¬ 
nication’; F = 15.19, p < .001 for 'Student Management’; F = 15.04, p < .001 for ‘Course Management/ 
Policy Knowledge’), except ‘Preparation for Feedback and Assessment’ (see Table 4). To further 
examine which specific groups differed, post-hoc tests (i.e., pairwise comparisons) were performed. 
Students (M= 4.45, SD=.44) rated significantly higher on 'Clear Communication’ than GTAs ( M = 
4.28, SD = .55) and faculty (M = 4.20, SD = .57) did. There was no significant difference between 
GTAs and faculty in this category. GTAs (M = 3.84, SD = .75) placed significantly higher importance 
on ‘Student Management’ than students (M = 3.43, SD = .79) and faculty (M = 3.40, SD = .74) did. 
As figure 1 indicates, the same pattern was observed for ‘Course Management/Policy Knowledge’, 
with GTAs (M = 4.05, SD = .59) rating higher than students ( M = 3.7, SD = .57) and faculty ( M = 
3.88, SD = .57). 
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Importance Competence 



GTAs 

Students 

Faculty 

F-test 

GTAs 

Students 

Faculty 

F-test 

FI 

4.28 (.55) 

4.45 (.44) 

4.21 (.57) 

4.31* 

3.79 (.74) 

3.47 (.81) 

3.31 (.68) 

11.66** 

F2 

3.84 (.75) 

3.43 (.79) 

3.40 (.74) 

15.19** 

3.57 (.79) 

3.42 (.75) 

3.12 (.56) 

6.39** 

F3 

4.42 (.54) 

4.37 (.44) 

4.27 (.59) 

1.40 

3.92 (.73) 

3.72 (.74) 

3.50 (.67) 

7 31** 

F4 

4.05 (.59) 

3.73 (.58) 

3.88 (.57) 

15.04** 

3.85 (.67) 

3.63 (.64) 

3.56 (.59) 

7.05** 


Note. *p < .05, **p < .01; Factorl=Clear Communication, Factor2 = Student Management, Factor3 = Preparation for 
Feedback and Assessment, Factor4 = Course Management/Policy Knowledge. 

Table 4. Difference among Groups in Importance and Competence Ratings: Means and 


Standard Deviations. 



Figure 7 . Consistent distributions of Importance average ratings by Groups (GTA, Student, 
and Faculty) for the four categories. 


In terms of the distribution of importance average ratings for the four categories, we found 
consistent patterns across the three groups. ‘Clear Communication’ and 'Preparation for Feedback 
and Assessment’ were perceived as being relatively more important than the other two categories, 
‘Student Management’ and ‘Course Management/Policy Knowledge’ (Figure 1). 

Difference among Groups in Competence Ratings 

On the competence ratings, subsequent univariate analysis of variance (ANOVA) indicated overall 
significant differences (F = 11.66, p < .001 for ‘Clear Communication’; F = 6.39, p < .01 for ‘Student 
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Management’; F = 7.31, p < .01 for ‘Preparation for Feedback and Assessment’; F = 7.05, p < .01 for 
‘Course Management/Policy Knowledge’) among the groups in all categories (Table 4). Post-hoc 
tests (i.e., pairwise comparisons) revealed that the significant differences occurred between GTAs 
and the other two groups. This pattern was consistent across the categories. Faculty and students 
rated GTA competence significantly lower than the GTAs did (Figure 2). 

In terms of the distribution of competence average ratings for the four categories, all partici¬ 
pants consistently perceived that GTAs are less competent in 'Clear Communication’ and ‘Student 
Management’ than they are in the other two categories (‘Preparation for Feedback and Assessment’ 
and ‘Course Management/Policy Knowledge’). 

Discrepancies between Importance and Competence Ratings 

We attempted to identify important but under-prepared GTA roles and responsibilities. We 
examined the degree to which each group showed discrepancies between importance and com¬ 
petence ratings. A paired-t test revealed that GTAs showed significant discrepancies between 
importance and competence ratings in all categories (Table 5). Students and faculty reported 
significant discrepancies in the ‘Clear Communication’ and ‘Preparation for Feedback and As¬ 
sessment,’ while both groups did not show significant discrepancies in the ‘Student Manage¬ 
ment.’ We found that two categories, ‘Clear Communication’ and ‘Preparation for Feedback 
and Assessment,’ were consistently perceived as being more important but less prepared GTA 
roles by all three groups. 
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GTAs 



Students 



Faculty 


IMP 

COM 

D 

t-test 

IMP 

COM 

D 

t-test 

IMP 

COM 

D 

t-test 

FI 

4.28 

3.79 

.49 

10.84** 

4.45 

3.47 

.98 

14.64** 

4.21 

3.31 

.90 

7 77 ** 

F2 

3.84 

3.57 

.27 

5.01** 

3.43 

3.42 

.01 

.29 

3.40 

3.12 

.28 

1.92 

F3 

4.42 

3.92 

.50 

9.91** 

4.37 

3.72 

.65 

11.15** 

4.27 

3.50 

.77 

7.08** 

F4 

4.05 

3.85 

.20 

4.76** 

3.73 

3.63 

.10 

1.81 

3.88 

3.56 

.32 

2.78** 


Note. *p < .05, **p < .01; IMP=Importance, COM = Competence, D = Discrepancy; Factorl = Clear Communication, 
Factor2 = Student Management, Factor3 = Preparation for Feedback and Assessment, Factor4 = Course Management/ 
Policy Knowledge 

Table 5. Discrepancies between Importance and Competence Ratings. 


DISCUSSION 


Survey validity 

We developed a needs assessment survey to measure GTAs’ perceived importance of various 
roles and responsibilities. Our results provide strong evidence that it is a valid and reliable measure. 
Based on the four-factor structure that an exploratory factor analysis indicated, we conceptualized 
four categories of GTA roles and responsibilities; 1) clear communication, 2) student management, 
3) preparation for feedback and assessment, and 4) course management /policy knowledge. 

The findings of the current study have theoretical and practical implications for GTA training. 
Theoretical implication is that GTAs’ roles and responsibilities are complex in nature and thus it is 
important to take multi-dimensional aspects of GTA roles into consideration in the measure of their 
needs for professional development. As shown in Table 2, GTAs placed different levels of importance 
on each category. GTAs tend to perceive managing and dealing with student behaviors and moti¬ 
vation problems (‘Student Management' category) as the area of least importance. This would be 
consistent with an expectation that such problems should not exist among students enrolled in an 
engineering discipline or in college generally, especially among the second-year or later students 
that most GTAs teach. In addition, GTAs may not be inclined to help those they perceive as less mo¬ 
tivated or properly engaged succeed, instead thinking that if the students don’t take responsibility 
they should not be pursuing a degree in engineering. If so, this may reflect the GTAs perception of 
themselves as motivated and interested. 

In contrast, GTAs felt highly responsible in areas such as being prepared to perform their primary 
roles such as grading student work and providing useful feedback (‘Preparation for Feedback and As¬ 
sessment’ category), and having an ability to effectively communicate with students to facilitate better 
learning (‘Clear Communication’). These factors may reflect the GTAs experiences as well. They may 


14 


WINTER 2013 












ADVANCES IN ENGINEERING EDUCATION 

A Survey to Capture Needs Assessment for GTA Training 



have received criticism for deficiencies in these areas or be critical of those that taught them and may 
differ in how they attribute differences in the quality of instruction to specific knowledge, habits, or 
skills. Providing useful feedback and consistent grading are very desirable characteristics of teachers. 
Herrington and Nakhleh (Herrington and Nakhleh 2003) found that when asked to rank GTA charac¬ 
teristics, GTAs and students considered GTA knowledge and skills more important than GTA attitudes 
toward students, with students ranking consistency in grading higher than GTAs did. Herrington and 
Nakhleh also found that GTAs and students ranked the GTAs’ role in motivating students low. 

These results are in direct contrast to the motivation behind GTA training discussed in the Back¬ 
ground and Rationale section. This clearly indicates that anyone who wants to develop a training 
program for the GTAs sampled here must emphasize the role that GTAs play in student development. 
The training should not only convey that GTAs are responsible for engaging and motivating students, 
but also provide sound pedagogical tips and exercises on how to engage and motivate students. 

Furthermore, correlation analysis yielded evidence for concurrent validity. We found that GTAs’ 
ratings of importance had a positive relationship with intrinsic and identified motivation to teach, 
suggesting that GTAs who enjoy and value what they are doing are highly likely to take more re¬ 
sponsibilities in all categories of GTA roles. The positive links between these variables provides not 
only further evidence of construct validity of the needs assessment survey but also provides strong 
rationale for addressing the four categories of GTA roles in teaching workshops to promote intrinsic 
interest and value of teaching. 

Importance and Competence Comparisons 

In addition to examining the psychometric properties of the survey, we aimed to provide useful 
information that the survey could provide in the design and development of GTA training program. 
We found significant differences in importance and competence ratings among GTAs, faculty, and 
students. The information allowed us to capture diverse needs, viewpoints, and perspectives about 
GTA preparedness from different groups involved in engineering education and to incorporate them 
into the development of GTA training program. 

One of the intriguing findings was that GTAs’ self-perception about their capabilities to perform 
teaching activities was positive and optimistic. Although positive beliefs in their competence is 
motivationally encouraging (Bandura 1997), inflated estimation of their ability may interfere with 
them expending more effort to develop their teaching competence. This concern is supported by 
the low ratings of faculty and students about GTA competence. 

Furthermore, the result from the current study that faculty and students tend to consider GTAs 
significantly less competent than GTAs consider themselves presents us with a challenge. Faculty often 
remain distant from the laboratories that the GTAs teach and therefore do not provide the guidance 
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the GTAs needs or desire (Dotger 2010). Thus faculty are not contributing to the development of the 
GTAs and helping to narrow this gap. We wonder whether presented with these data, faculty will come 
to the realization and take personal action or will they remain distant and suggest that others (e.g. 
lab coordinators, teaching and learning centers) be required to take more action. Alternatively, for 
those faculty who think that teaching ability is an intrinsic trait (Svinicki 2004), we would predict no 
interest or support for programs to help GTAs develop skills. The survey results would help develop 
a strategy for using this information to inform GTAs of the gap and encourage appropriate reflection 
on their part so as to motivate them toward mastery learning. This also provides us with a baseline 
against which to measure changes after the intervention of a GTA training program. 

Unlike the competence ratings, we found similar patterns in the distribution of responses regard¬ 
ing the importance ratings although there were significant mean differences among the groups. All 
participants including GTAs, faculty and students perceived 'Clear Communication’ and ‘Prepara¬ 
tion for Feedback and Assessment’ as more important GTA roles and responsibilities than ‘Student 
Management’ and ‘Course Management/Policy Knowledge.’ 

Combined with this finding, the results on discrepancies between the importance and competence 
ratings by the three participant groups offers more specific examples of how the survey results can 
be used in developing a GTA training program. The discrepancy scores between importance and 
competence ratings can be used to identify the primary focus of GTA training. In the present study, 
particularly two categories of GTA roles were consistently identified across different groups as train¬ 
ing areas that require more attention. While GTAs’ ability to communicate clearly and the extent to 
which they are prepared to provide adequate feedback and assessment were commonly considered 
more important by GTAs, faculty and students, we found that GTAs were not perceived as adequately 
skilled or prepared for these areas. Our data provided evidence for the utility value of using our needs 
assessment survey for identifying important but under-prepared GTA roles/responsibilities. 

The survey enables the training to be scaled with regards to time and resources available based 
on the results. For example, for the set of results presented here, if one had very limited resources, 
they would cover only the two factors: ‘Clear Communication’ and ‘Preparation for Feedback and 
Assessment.’ However, if resources were available, all four factors deserve attention due to the com¬ 
petency discrepancy. Thus, the survey provides a basis for prioritizing the topics to be covered. 


CONCLUSIONS, LIMITATIONS, AND FUTURE WORK 

We conducted the research described in this paper to ascertain, through surveys of faculty, GTAs, 
and students, what characteristics are considered to be most important in successful engineering 
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GTAs and the perceptions of GTAs’ competency levels. This study was undertaken in preparation for 
the development and implementation of a GTA training program, both to set the objectives of the 
program and to establish a baseline for measuring the impact of the program in the future. We tested 
psychometric properties of the measure by examining factor structure, reliabilities and concurrent 
construct validity. To examine concurrent construct validity, we examined how our new survey is related 
to teacher motivation measure, an empirically validated measure. Based on our results, we conclude 
that the survey showed favorable psychometric properties and offered beneficial information for the 
development of TA training program at our institution. Other notable findings include the following: 

• There are 4 clear factors based on the importance ratings: Clear Communication, Student 
Management, Preparation for Feedback and Assessment, and Course Management/Policy 
Knowledge. 

• In terms of the distribution of importance average ratings for the four categories, we found 
consistent patterns across the three groups. ‘Clear Communication’ and ‘Preparation for Feed¬ 
back and Assessment' were perceived as being relatively more important than the other two 
categories, ‘Student Management’ and ‘Course Management/Policy Knowledge.’ 

• Post-hoc tests (i.e., pairwise comparisons) for competence ratings revealed that significant 
differences occurred between GTAs and the other two groups. This pattern was consistent 
across the categories. Faculty and students rated GTA competence significantly lower than 
the GTAs did. 

• In terms of the distribution of competence average ratings for the four categories, all par¬ 
ticipants consistently perceived that GTAs are less competent in ‘Clear Communication’ and 
‘Student Management’ than they are in the other two categories (‘Preparation for Feedback 
and Assessment’ and ‘Course Management/Policy Knowledge’). 

Although the survey showed favorable psychometric properties, further testing is warranted to 
confirm that the new measure of GTA’s needs assessment can be used as a reliable and valid tool 
across institutions. One of the limitations of the study was the low number of female respondents 
among the GTAs, and an overall low number of respondents among the faculty. 

The survey allows various interest groups to customize their own program of training based on 
the survey responses. Appropriately customized GTA training helps maximize its effectiveness and 
increases the impact on GTAs’ professional development. It is thus hoped that other institutions 
adopt the survey to guide their GTA training efforts. 

The survey used (GTA version) is available at http://www.survevmonkev.eom/s/TA . Surveys 
for faculty and students are at http://www.survevmonkey.eom/s/facultv-survev and http://www. 
survevmonkey.com/s/students-survev respectively. Faculty that wish to adapt the survey for their 
institutions are encouraged to contact the authors of this paper. 
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